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Abstract 

Multi-component interventions for conduct disorder target several contexts of a child’s life (e.g., 
both home and school environments) and are generally more effective than single- component behavioral 
interventions. Whether the multi- component approach is cost-effective remains an unanswered question. 
This article analyzes two decades of data from the Incredible Years (lY) Series to examine the cost- 
effectiveness of delivering multiple, stacked intervention components versus a single-component delivery 
approach. Cost-effectiveness analysis (CEA) provides decision makers with important economic 
information that can be used to aid in the selection of a program delivery format from one of several 
competing approaches. CEA concepts, including explicit budget constraints and strict dominance, are 
demonstrated using lY data; guidelines for interpreting CEA results are provided. Our analyses suggest 
that combining intervention components is a cost-effective approach to treating behavioral problems in a 
clinic-based youth population. 

Keywords: cost-effectiveness, multi-component interventions. Incredible Years Series, conduct disorder. 


Conduct disorder (CD) is among the most common emotional and behavioral disorders affecting 
youth (Institute of Medicine, 1989), occurring in an estimated 10 percent of preschool and early-school- 
age children (Institute of Medicine, 1989; Webster-Stratton & Reid, 2003). Children with CD often 
engage in a variety of behaviors that are detrimental to themselves, their families, and society (Institute of 
Medicine, 1989; Webster-Stratton & Reid, 2003). Youth diagnosed with CD exhibit persistent behaviors 
consistent with three or more of the following four behavioral categories: aggression toward people or 
animals; property destruction; deceitfulness or theft; and serious rule violations (DSM-IV, 1994). These 
children frequently lack critical social and self-regulation skills necessary for making friends, and, as a 
result, often endure peer rejection and isolation from an early age (Kaiser & Hester, 1997; Miller- 
Johnson, Coie, Maumary-Gremaud, Bierman, & CPPRG, 2002). 

CD puts youth at risk for other costly outcomes such as weapon use, alcohol use, and other drug use 
and increases the likelihood of teenage pregnancy, dropping out of school, and police contact (Robins and 
Price, 1991; Achenbach, Howell, McConaughy, & Stanger, 1998; Bardone, Moffitt, Caspi, Dickson, 
Stanton, & Silva, 1998; Scott, 1998). Scott, Knapp, Henderson, and Maughan (2001) found evidence of a 
linear relationship between severity of conduct problems and societal costs; children with conduct 
problems (but who did not meet diagnostic criteria for CD) and children diagnosed with CD generated 
societal costs three and ten times greater than children with no conduct problems, respectively. Children 
with CD often become involved with public, child-serving systems, such as juvenile justice. This 
involvement creates juvenile court costs, incarceration costs, costs of lost productivity due to 
incarceration, and victim costs (both tangible and intangible) all home by society (Cohen, 1998). Cohen 
(1998) estimates that one life of crime results in societal costs ranging from $1.3 to $1.5 million. 

Research suggests that early intervention is most effective, especially for those at greatest risk. 
Although the incidence of CD is higher among adolescents (Searight, Rottnek, & Abby, 2001), a small 
minority of young children begin to display conduct disorder symptoms as early as preschool age 
(Moffitt, 1993). Without intervention, childhood-onset CD generally results in bleaker trajectories than 
adolescent-onset CD (Searight et al., 2001). Compared with their peers, children identified in 


284 


JEIBI 


3 . 4 - 4.1 


kindergarten as high-risk for behavioral problems are more likely to be placed on medication, to repeat a 
grade, to receive mental health services, and to come in contact with police (Jones, Dodge, Foster, Nix, & 
CPPRG, 2002). Early and effective CD interventions are essential for improving both individual and 
societal outcomes. Behavioral prevention and intervention programs designed for children with CD may 
also benefit children with or at risk for the development of other mental health conditions. For example, 
Lahey, Loeber, Burke, Rathouz, and McBumett (2002) found that CD is both co-morbid with and 
predictive of ADFID, oppositional defiant disorder (ODD), depression, and anxiety. Therefore, efforts to 
prevent and treat CD may also prevent or improve related psychopathology. 

Past research has identified numerous etiologic factors that contribute to the development of CD; 
these factors include (but are not limited to) child, parent, and environmental characteristics. As a result, 
research suggests that a comprehensive intervention approach is the most effective means for treating CD 
(e.g., Craig & Digout, 2003). Multi-component treatment approaches target multiple risk factors across 
several contexts of a child’s life (Kaiser & Flester, 1997; Craig & Digout, 2003). Typically, multi- 
component behavioral interventions focus on the home and school settings, and target parent, teacher, and 
peer communication skills. Because of the various components involved in multi-component 
interventions, health decision makers are able to choose from several different implementation 
approaches. For example, an agency might elect to implement one, several, or all components of a multi- 
component behavioral intervention. Agencies should select the most cost-effective treatment combination 
for implementation; however, the guidelines for selecting the most cost-effective treatment approach are 
often overlooked. 

Although a combination of treatment components may reduce negative behaviors most reliably, 
agencies typically implement just one component. Often, the selected component is either logistically the 
most feasible to implement or has the lowest per-child treatment cost. Schools may opt for behavioral 
interventions that involve both teachers and children; they may be less likely to engage parents due to cost 
or logistical difficulties. Similarly, mental health agencies are more likely to involve parents in the 
treatment process; however, they are less likely to engage teachers due to similar involvement difficulties. 
In such situations, agencies may miss opportunities to implement more cost-effective interventions. If 
agencies fail to consider the cost-effectiveness of implementation strategies, limited resources may not be 
allocated optimally. A higher effect size for a multi-component treatment may justify its increased cost. 

On the other hand, if the effect size for a more expensive multi-component treatment is only slightly 
higher than that of single-component interventions, agencies may be better off treating comparatively 
more children using the single component that produces the greatest behavioral improvements. Therefore, 
the implementation decision should consider not only treatment costs, but outcomes and the specific 
population to be treated, as well. Simply selecting the implementation strategy that is the most feasible or 
that has the lowest per-person costs creates situations in which agencies may miss out on maximizing 
health gains for marginal increases in cost (Bala & Zarkin, 2002). 

While the effectiveness of multi-component CD interventions has been demonstrated, their cost- 
effectiveness is largely unknown. This article considers the differential cost-effectiveness of delivering 
treatment components in combination. Our perspective is that of the public health official with a fixed 
budget for prevention programs. We also provide an introduction to cost-effectiveness methodology and 
consider common financial issues that arise when health decision makers must choose from a series of 
competing treatment combinations of varying cost, intensity, and complexity. This framework is used to 
evaluate the cost-effectiveness of The Incredible Years Parents, Teachers, and Children Training Series - 
an evidence-based multi-component intervention created to treat young children with early-onset conduct 
problems (CD/ODD). 
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Prior Research 


Treating Conduct Disorder 

Effective and early behavioral intervention must be provided before patterns of negative 
behaviors become habitual (e.g., Kaiser & Hester, 1997; Webster-Stratton & Reid, 2003). Therefore, 
interventions designed for very young children may be more successful in treating CD than those 
designed for school-age children (Keenan & Wakschlag, 2000). A large body of research currently 
indicates that multi-component interventions are most effective for treating CD in youth-based 
populations (e.g., Craig & Digout, 2003). 

One example is the Incredible Years Series; it has been identified as an effective 
CD/ODD treatment and prevention program for young children by the Office of Juvenile Justice 
and Delinquency Prevention (Webster-Stratton, 2000). Additionally, when an independent APA 
review committee reviewed findings from over 82 studies of CD interventions, the Incredible 
Years Series and programs based on the manual Living With Children (Patterson & Gullion, 
1968) were reported as the only two behavioral intervention strategies that met the criteria for 
well-established efficacious CD treatments (Brestan & Eyberg, 1998). 

Cost-Effectiveness Evaluation of Social Interventions 

Economic evaluation can take any of several forms. Perhaps better known, benefit-cost analysis 
involves measuring both the costs and the benefits of a program in dollar terms (Thompson, 1980). Cost- 
effectiveness analysis (CEA), on the other hand, does not assign monetary costs to all benefits; program 
costs are assigned monetary values while program benefits are valued in non-monetary units (e.g., health 
indicators) (Johannesson, 1995). CEA requires fewer research dollars but does not provide a “bottom 
line” for each program; rather, it is useful for comparing competing programs and well-suited to the 
question addressed here. 

Recent interest in the scope and efficiency of health prevention programs, as well as limited funding 
for health interventions, has stimulated cost-effectiveness research within many public health sectors. 
Studies incorporating CEA commonly focus on health prevention programs such as those designed to 
decrease transmittal rates of HIV/AIDS, increase smoking cessation initiatives, and avert costly side 
effects associated with diabetes and depression (e.g., Pinkerton, Holtgrave, Johnson-Masotti, Turk, Hackl, 
DiFranceisco, et al., 2002; Song, Raftery, Aveyard, Hyde, Barton, & Woolacott, 2002; Hoerger, Bethke, 
Richter, Sorensen, Engelgau, Thompson, et al., 2002; Raikou, Gray, Briggs, Stevens, Cull, McGuire, et 
al., 1998; Scott, Palmer, Paykel, Teasdale, & Hayhurst, 2003; Miller, Chilvers, Dewey, Fielding, Gretton, 
Palmer, et al. 2003). 

The current study applies CEA methodology to data generated from repeated implementations of the 
Incredible Years Series. Given the multiple implementation formats available for this program, CEA data 
will be combined with additional information to help inform health decision makers as to which 
Incredible Years treatment format would be most cost-effective given their agency’s unique budget 
constraints and intervention goals. 

The Incredible Years Parent, Teacher and Children ’s Series: Program Design and Goals 

Program History and Goals. The Incredible Years Parents, Teachers, and Children Training Series - 
developed by Carolyn Webster-Stratton, Ph.D., and evaluated by colleagues at the University of 
Washington’s Parenting Clinic - is a multi-component program designed to treat young children (ages 3 
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to 8) with early-onset conduct problems. Based on implementation methods, the Incredible Years (lY) 
Series also has been adapted to serve as a cost-effective, community-based prevention program for 
children at risk for the development of CD. Over the past 20 years, this intervention has been repeatedly 
implemented in both clinic and natural environment contexts such as mental health settings and schools. 

Ultimately, the lY Series strives to prevent delinquency, drug abuse, and violent acts among high-risk 
children. However, immediate goals of the program include the reduction of conduct problems in 
children; the enhancement of social, emotional, and academic capabilities of children; the promotion of 
parental competence and positive discipline strategies; the strengthening of families as well as the school- 
home connection; and the enhancement of teacher classroom management skills (Webster-Stratton, 

2000). 

The lY Series is comprised of three main single treatment components each focusing on different 
contexts and types of social interaction a child encounters in his or her daily life. The three treatment 
components include (1) a child-based program (referred to as Child Training or CT); (2) a parent-based 
program (referred to as Parent Training or PT); and (3) a teacher-based program (referred to as Teacher 
Training or TT). CT and PT leaders initially learn program curricula from certified lY trainers; following 
training, CT and PT leaders deliver program curricula to child and parent participants, respectively, over a 
series of weekly small group sessions. Teachers taking part in TT initially learn the program content 
during a 4-day long training workshop led by certified lY trainers; trained teachers then incorporate 
program content into their daily classroom activities over the course of the school year. The lY Series, 
therefore, contains both selective and non-selective treatment components; CT and PT focus specifically 
on high-risk or diagnosed children while TT also offers treatment benefits to the classmates of high-risk 
or diagnosed children. For a detailed description of treatment component goals, curriculum, and 
implementation methods, please see Webster-Stratton (2000). 

Webster-Stratton and colleagues have implemented the lY Series using the three single treatment 
components either alone (e.g., CT program alone) or stacked in various combinations (e.g., CT plus TT 
and/or PT). Different combinations of the lY components are recommended depending on the targeted 
child population. 

Program Success. The lY Series has been effective in reducing the frequency of children’s conduct 
problems regardless of treatment locale. Service agencies (mental health agencies, child welfare systems, 
and schools) continue to implement the lY Series; large-scale diffusion of the program has occurred 
across the United States, Canada, UK, and Norway. Agencies adopting the lY Series are responsible for 
budgeting for initial training from certified lY trainers, program materials (videotapes, group leader 
manuals, parent and child materials, and handouts), program implementation, and ongoing consultation 
with lY trained staff. Following the initial materials and training fees, the lY Series may be offered to 
participants from successive cohorts at minimal cost to the service agency. 

Past literature has assessed the impact of participant characteristics, individual component intensity, 
and multi-component delivery methods on the effectiveness of the lY Series. Numerous randomized 
control group studies by the developer (e.g., Webster-Stratton, 1990; Webster-Stratton & Hammond, 

1997; Webster-Stratton & Reid, 1999a; Webster-Stratton & Reid, 1999b; Webster-Sratton, Reid & 
Hammond, 2001) and by independent investigators (e.g., Taylor, Schmidt, Pepler, & Hodgins, 1998; 
Miller & Rojas-Flores, 1999; Scott, Spender, Doolan, Jacobs, & Aspland, 2001; Barrera, Biglan, Taylor, 
Gunn, Smolkowski, Black, et al., 2002) strongly support the assertion that the lY Series consistently 
improves child behavior across a range of outcome indicators. However, no investigation into the cost- 
effectiveness of stacking lY intervention components has been completed to date. 
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This paper utilizes eost and outeome data from the lY Series to examine the eost-effeetiveness of 
staeking multiple intervention eomponents versus delivering single intervention eomponents. Traditional 
CEA theory is employed, along with eonventional CEA deeision eriteria, to produee finaneial data that 
offer insight into the eeonomie appropriateness of various lY implementation strategies. This paper serves 
as the first study to date of the eost-effeetiveness of staeked eomponents within the eontext of the lY 
Series. 


Method 


Participant Characteristics 

Data were eombined from 2 1 separate eohorts enrolled in six randomized elinieal trials of the lY 
Series (Webster-Stratton, 1982; 1984; 1994; Webster-Stratton & Hammond, 1997; Webster-Stratton, 
Hollinsworth, & Kolpaeoff, 1989; Webster-Stratton & Reid, 1999a). The final sample ineluded 459 
ehildren, ages 3-8, who had partieipated in lY Series researeh over the past 20 years. Data from these 
studies eould be pooled beeause of eommon data eolleetion proeedures; all six studies measured program 
effieaey using an identieal set of ehild behavior measures. Random assignment and longitudinal follow-up 
oeeurred with eaeh elinieal trial. 

The following eriteria were required for entry into the lY elinieal-based treatment-outcome trials: (1) 
the child was between 3 and 8 years of age; (2) the child had no debilitating physical impairment, 
intellectual impairment, or history of psychosis and was not already receiving psychological treatment; 

(3) the primary clinic referral reason was for conduct problems such as noncompliance, aggression, and 
oppositional behavior that continued for more than six months; (4) parent-report symptoms on the Eyberg 
Child Behavior Inventory (ECBI) were clinically significant (more than two standard deviations above 
the mean); and (5) the child met criteria for ODD and/or CD according to either the Diagnostic and 
Statistical Manual of Mental Disorders, Third Edition, Revised (DSM-III-R, 1986) or the DSM-IV (1994) 
depending on the child’s study entry date (Webster-Stratton & Reid, 2003). 

Following baseline assessments, families were assigned to one of seven conditions: (1) Child 
Training only (CT); (2) Parent Training only (PT); (3) Child Training and Parent Training (CT-l-PT); (4) 
Parent Training and Teacher Training (PT-l-TT); (5) Child Training and Teacher Training (CT-l-TT); (6) 
Child Training, Parent Training, and Teacher Training (CT-l-PT-l-TT); and (7) a control condition. A more 
detailed summary of participant characteristics may be found below in Table 1. 

Table 1: Participant Summary Statistics by Incredible Years Treatment Category 


Treatment 

Category 

N 

Child Ethnicity 

Average Child's 
Age (Months) at 
Intake 

Average Mother's 
Age ( Y ears) at Child 
Intake 

CT 

54 

Caucasian: 48 




Boy: 43 
Girl: 1 1 

Hispanic: 0 
Black: 4 
Other: 2 

72.3 

36.1 

PT 

292 

Caucasian: 265 




Boy: 215 
Girl: 77 

Hispanic: 3 
Black: 4 
Other: 20 

59.6 

34 
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CT+PT 

38 

Boy: 27 
Girl: 1 1 

Caucasian: 31 
Hispanic: 1 
Black: 2 
Other: 4 

72.4 

35.4 

PT+TT 

24 

Boy: 22 
Girl: 2 

Caucasian: 21 
Hispanic: 0 
Black: 1 
Other: 2 

67.4 

38.3 

CT+TT 

11 

Boy: 9 
Girl: 2 

Caucasian: 7 
Hispanic: 1 
Black: 0 
Other: 3 

74.3 

35.6 

CT+PT+TT 

19 

Boy: 17 
Girl: 2 

Caucasian: 16 
Hispanic: 1 
Black: 0 
Other: 2 

71 

39.9 

Control 

21 

Boy: 19 
Girl: 2 

Caucasian: 18 
Hispanic: 0 
Black: 0 
Other: 3 

68.9 

36.1 

Total 

459 

Boy: 352 
Girl: 107 

Caucasian: 406 
Hispanic: 6 
Black: 11 
Other: 36 

69.4 

36.5 


Estimating Treatment Costs 

To estimate per-ehild costs of the lY treatment combinations (excluding the control condition), total 
per-child costs were first estimated for each of the three lY single components (CT, PT, and TT). Costs 
were derived using a series of detailed financial estimates provided by the developer of the lY Series. The 
payer perspective was used to determine costs; that is, these financial estimates included all fees for 
which the agency implementing the lY Series is responsible. Estimates included fees associated with 
training and ongoing supervision of CT and PT group leaders and trained teachers participating in TT; 
group leader salary including time for peer review, self-study, and preparation; costs of providing 
materials for participants; and additional fees - both on- and off-site - necessary for actual program 
implementation (i.e., on-site childcare, participant meals, cab vouchers, and off-site childcare 
compensation). These costs comprise those described by Weinstein (1990) as direct treatment costs and 
direct personal costs stemming from program implementation. CEA ratios generated in this paper reflect 
financial estimates based on 2003 dollars. 

After total per-child costs were estimated for each of the three single components (CT, PT, and TT), 
estimates were summed to generate total per-child costs for each of the four stacked treatment 
combinations (CT+PT, PT+TT, CT+TT, and CT+PT+TT) (i.e., total per-child cost of CT+PT = total per- 
child cost of CT + total per-child cost of PT). Table Al, located in Appendix A, summarizes how per- 
child costs were estimated for each treatment category. 
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Cost Estimate Assumptions. Total per-child cost estimates did not include costs associated with the 
space required for initial leader and teacher training and weekly small group sessions. It is assumed that 
agencies and schools implementing lY will provide on-site space in which group leader and teacher 
training, as well as group sessions, may be conducted. (If an agency does not have space available for 
training and small group sessions, the cost of space rental should be included when calculating total per- 
child cost estimates). It should also be noted that the CT+PT condition required the purchase of only one 
set of parent manuals at the cost of $179.40 per 12 parents; therefore, this fee was not duplicated when 
summing total per-child costs for CT and PT to derive the total per-child cost for CT+PT. 

Various cost categories included in the total per-child cost estimates represented one-time program 
initiation costs. For example, estimates presented in Table A1 assumed that each new CT group leader, 

PT group leader, and TT trained teacher complete just one sequence of the lY Series following 
certification. However, in real-world implementation, newly trained group and classroom facilitators who 
have completed training in CT, PT, and TT will likely lead more than one sequence of the lY Series. 
Because group leaders and teachers complete training only prior to the first lY sequence, training costs 
depreciate as the number of children participating in lY increases. Similarly, after the first sequence of 
lY, costs associated with one-time purchases of materials are not included in additional sequences of lY. 
Therefore, with each additional cohort of children treated, total per-child costs decrease for each treatment 
category. For these reasons, total per-child treatment costs presented in Table A1 can be considered 
conservative. 

Table Al: Mean Per-Child Costs by Incredible Years Treatment Category 


Treatment Combination PT+TT 



CT 

PT 

CT+PT 

PT 

TT 

Trainine Fees 

CT Leader Training by Cert. lY Trainer (3 8-hour days) 

$3,600 


$3,600 



PT Leader Training by Cert. lY Trainer (3 8-hour days) 
TT Teacher Training by Cert. lY Trainer (4 8-hour days) 


$3,600 

$3,600 

$3,600 

$4,800 

CT Leader-in-Training's Time (3 8-hour days) 

$480 


$480 



PT Leader-in-Training's Time (3 8-hour days) 
TT Teacher-in-Training's Time (4 8-hour days) 


$480 

$480 

$480 

$400 

Material Fees - (Trainine and Small Groun Session) 

CT Small Group Session Materials 

$975 


$975 



CT Puppet 

$269 


$269 



CT Leader Lesson Plans 

$150 


$150 



CT Small Group Session Handouts (for 6 children) 

$14 


$14 



PT Training Materials 


$15 

$15 

$15 


PT Small Group Session Materials 


$1,300 

$1,300 

$1,300 


PT Leader Manual 


$90 

$90 

$90 


CT and PT Parent Manuals (for 12 parents) 
TT Teacher Handbook 

$179 

$179 

$179 

$179 

$25 

TT Classroom Handouts 





$10 

Additional Fees - Staff Time 

CT Leader's Time in Sessions 

$880 


$880 



CT Weekly Supervision 

$440 


$440 
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PT Leader's Time in Sessions $720 $720 $720 

PT Leader Additional Time $480 $480 $480 

TT Consultation Costs $390 


Additional Fees - lY Imnlementation 

PT Small Group Session Meals 
PT Small Group Session Babysitting Fees 
PT Small Group Session Cab Vouchers 
PT Small Group Session Day Care Costs 
TT Training Session Snacks 


$1,000 

$1,080 

$240 

$288 

$1,000 

$1,080 

$240 

$288 

$1,000 

$1,080 

$240 

$288 

$160 

Total 

$6,987 

$9,472 

$16,280 

$9,472 

$5,785 

Divided by Number of Children Served 

6 

6 

6 

6 

20 

Adjusted Total (Per-Child Cost) 

$1,164 

$1,579 

$2,713 

$1,579 

$289 

Total Per-Child Cost for Treatment Category 

$1,164 

$1,579 

$2,713 

$1,868 


Treatment Combination 

CT +TT 

CT-l-PT-l-TT 


CT 

TT 

CT+PT 

TT 

Trainine Fees 

CT Leader Training by Cert. lY Trainer (3 8-hour days) 

$3,600 


$3,600 


PT Leader Training by Cert. lY Trainer (3 8-hour days) 
TT Teacher Training by Cert. lY Trainer (4 8-hour days) 


$4,800 

$3,600 

$4,800 

CT Leader-in-Training's Time (3 8-hour days) 

$480 


$480 


PT Leader-in-Training's Time (3 8-hour days) 
TT Teacher-in-Training's Time (4 8-hour days) 


$400 

$480 

$400 

Material Fees - (Trainine and Small Groun Session) 

CT Small Group Session Materials 

$975 


$975 


CT Puppet 

CT Leader Lesson Plans 

$269 

$150 


$269 

$150 


CT Small Group Session Flandouts (for 6 children) 

$14 


$14 


PT Training Materials 

PT Small Group Session Materials 



$15 

$1,300 


PT Leader Manual 



$90 


CT and PT Parent Manuals (for 12 parents) 
TT Teacher Flandbook 

$179 

$25 

$179 

$25 

TT Classroom Flandouts 


$10 


$10 

Additional Fees - Staff Time 

CT Leader's Time in Sessions 

$880 


$880 


CT Weekly Supervision 
PT Leader's Time in Sessions 

$440 


$440 

$720 


PT Leader Additional Time 
TT Consultation Costs 


$390 

$480 

$390 

Additional Fees - lY Imnlementation 

PT Small Group Session Meals 



$1,000 


PT Small Group Session Babysitting Fees 
PT Small Group Session Cab Vouchers 



$1,080 

$240 


PT Small Group Session Day Care Costs 
TT Training Session Snacks 


$160 

$288 

$160 
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Total 

Divided by Number of Children Served 
Adjusted Total (Per-Child Cost) 

Total Per-Child Cost for Treatment Category 


$6,987 $5,785 

6 20 

$1,164 $289 

$1,454 


$16,280 $5,785 

6 20 

$2,713 $289 

$3,003 


Calculating Treatment Outcomes 

Analyses involved pre-test and immediate post-test assessments for two key outcomes: (1) a teacher- 
reported Total Problem Behavior Score measured by the Behar Preschool Behavior Questionnaire (PBQ) 
and (2) a combined 5-item Negative Child Behavior Score measured by independent home observations 
according to the Dyadic Parent-Child Interactive Coding System - Revised (DPICS-R). The PBQ (Behar, 
1977) identifies children as young as age 3 who display symptoms of emotional problems. The Total 
Problem Behavior Score was formed by combining data from 36 teacher-reported items measuring three 
negative behavioral constructs: (1) hostile-aggressive; (2) anxious-fearful; and (3) hyperactive- 
distractible. The DPICS-R records behaviors of children and their parents in a home setting (Reid, 
Webster-Stratton, and Baydar, 2004). Thirty-nine parental and eight child behavioral categories are 
assessed during 30-minute in-home observations by third-party observers (Reid et al., 2004). The 
Negative Child Behavior Score was formed by combining data from five separate negative behavior 
measures: (1) negative physical actions, (2) destructive behaviors, (3) yell/cry/whine, (4) “smart talk”, 
and (5) overall behavior valence (Beauchaine, Webster-Stratton, and Reid, 2005). (Not all children 
participating in lY were enrolled in school at the time of treatment; therefore, the sample size for the 
Behar analysis was reduced somewhat.). 

The PBQ and DPICS-R outcome measures assess children’s behaviors across settings, capturing 
treatment impact on problem behaviors in the school and home environments. These two outcome 
measures also incorporate evaluations of child behavior by adults other than the parents of target children; 
therefore, these observational measures may represent more impartial views of children’s behaviors. 

Mean difference scores were created for both outcome variables by subtracting each child’s post-test 
score from their pre-test score and averaging within treatment category. Because both outcome measures 
code negative child behavior highly, post-test scores were expected to be lower than pre-test scores; lower 
post-test scores indicate that the lY treatment categories reduced the frequency of negative child behavior. 
Pre-post difference scores for each treatment category were then standardized by subtracting the mean 
pre-post difference score for the control condition and dividing by the standard deviation of the pre-test 
control group score. One-sided t- tests were performed to determine whether treatment categories’ 
difference scores were significantly different from zero. 

The Role of Agency Budget Constraints 

Agencies typically must operate within some form of financial constraints. The nature of an 
agency’s budget constraints plays an important role in determining which implementation strategy among 
alternatives is considered most cost-effective. Our analyses considered the scenario of public health 
officials with fixed, or explicit, budget constraints. 

Explicit budget constraints are defined as the specific per-person dollar amount (D) available for 
treating a condition in a target population (e.g., a health system has $50,000 to treat ADHD in a clinic- 
based population of 100 young children; D=$500) (Bala & Zarkin, 2002). This scenario is likely to be 
relevant to many health decision makers due to availability of funding. For example, small agencies, such 
as individual schools or local health departments are typically limited by a scarcity of public school 
system or local government funds. It is plausible that such agencies have a specific dollar amount 
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reserved for interventions, and onee this money is exhausted, no additional funds are available for 
treatment purposes. 

Economists often consider another financial situation - that of the implicit budget constraint. Bala and 
Zarkin (2002) define an implicit budget constraint as the maximum amount (A.) that an agency is willing 
to pay for a one-unit gain in treatment effectiveness (e.g., a health system is willing to pay $5,000 per new 
case of breast cancer detected in its early stages). Agencies limited by implicit budget constraints often 
have more financial flexibility when it comes to implementing interventions; under implicit budget 
constraints, agencies essentially choose a price that they are willing to pay per one-unit gain in 
effectiveness, regardless of total per-person expenditures. As a result, agencies limited by implicit budget 
constraints are able to fund more effective interventions that may be higher in per-person total costs, 
provided that the intervention produces unit gains in effectiveness at costs lower than A. For this reason, 
Bala and Zarkin (2002) state that agencies limited by implicit budget constraints are more able to keep 
pace with burgeoning technology compared to agencies limited by explicit budget constraints. For the 
purposes of this paper, we do not consider the implicit budget constraint scenario. 

When decision makers are limited by explicit budget constraints, the choice of a cost-effective 
treatment combination is made based on two types of cost-effectiveness ratios. Average cost-effectiveness 
ratios (ACERs) provide insight into the cost for which a given treatment produces a one-unit change in 
outcome. For the purposes of this paper, a treatment’s ACER indicated the cost for which the given lY 
treatment category produced a one standard deviation decrease in negative child behavior as measured by 
the two outcome measures previously described. However, depending on the nature of the health 
intervention, treatment ACERs may represent the cost per child vaccinated, the cost per death averted, or 
the cost per case of breast cancer detected. The formula for generating the ACER for hypothetical 
Treatment A is summarized below: 

Treatment A' s Average Total Cost Per Child (Ca) 

A.Cy£l\A — 

Treatment A's Average Total Effectiveness Per Child {Ea) 

Suppose Treatment A costs $1,100 per person and results in a per-person average of 0.22 standard 
deviation reduction in behavioral problems measured by a problem behavior outcome measure. The 
ACER for Treatment A would be $5,000 ($1,100 / 0.22), reflecting that, on average. Treatment A costs 
$5,000 to produce a one standard deviation reduction in problem behaviors. 

Health decision makers also rely on a second category of ratios, known as incremental cost- 
effectiveness ratios (ICERs). Whereas ACERs provide information regarding the cost-effectiveness of 
one treatment, ICERs compare the incremental cost-effectiveness of one treatment relative to another. 
ICERs are expressed as monetary values and represent how much more an agency would pay per unit of 
effectiveness beyond that produced by the less effective treatment were they to implement the more 
effective treatment of the pair (Bala & Zarkin, 2002). For example, consider two treatments - Treatment 
X and Treatment Y. Treatment Y is both more effective and more expensive than Treatment X. ICERxy 
indicates the price an agency would pay per unit of effectiveness above and beyond that produced by 
Treatment X if they were to implement Treatment Y. Therefore, whereas ACERs focus on the cost and 
effectiveness of a single treatment, ICERs focus on whether a treatment that is both more expensive and 
effective than another treatment achieves its additional effectiveness at reasonable costs. ICERs are of 
primary importance to agencies limited by implicit budget constraints (Bala & Zarkin, 2002). However, 
such ratios are also important when agencies are limited by explicit budget constraints. 


Identifying Strictly Dominated Treatment Categories 
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Information concerning treatment combinations’ per-person cost and effectiveness allows health 
decision makers to identify occurrences of strict dominance among a series of potential treatment 
combinations. As described by Drummond, Stoddart, and Torrance (1997), one treatment strictly 
dominates another if it produces better results at lower costs. Strictly dominated treatment approaches are 
immediately eliminated from further consideration by health decision makers since one or more programs 
exist that are both cheaper and more effective. Therefore, identifying strictly dominated programs helps 
decision makers narrow down the field of choices from which they will choose an implementation format. 

Economists often consider a second type of dominance among treatment alternatives. Extended 
dominance occurs when ICERs do not increase along with increasing program effectiveness (Bala & 
Zarkin, 2002; Drummond et al., 1997). An extendedly dominated program has an ICER higher than that 
of the next most effective program; in other words, an extendedly dominated program produces additional 
unit gains in effectiveness at incremental costs higher than that of the next most effective program. A 
decision to implement an extendedly dominated treatment means that an agency has missed an 
opportunity to achieve incremental gains in effectiveness at lower costs. Extended dominance becomes 
especially important in cases where health decision makers are limited by implicit budget constraints. 
However, for the purposes of this paper, only occurrences of strict dominance will be analyzed. 

Results 

Determining the Most Cost-Effective lY Implementation Strategy 

When limited by explicit budget constraints, an agency should choose the implementation 
strategy that maximizes effectiveness without exceeding available budget constraints. It is helpful to 
illustrate treatments’ costs and effectiveness, as well as agencies’ budget constraints, graphically when 
choosing an implementation strategy. As noted by Bala and Zarkin (2002), graphing per-person cost and 
effectiveness data for competing programs allows health decision makers to easily recognize how budget 
constraint changes would affect implementation decisions. Graphs are also helpful for visually 
representing ACERs and ICERs of competing programs. 

Treatment data are plohed graphically with the X-axis representing per-participant effectiveness 
and the Y -axis representing per-participant cost. The slope of any line segment connecting a program to 
the origin represents that program’s ACER; the slope of any line segment joining two programs 
represents the ICER for that particular pair of treatments (Bala & Zarkin, 2002). A curve known as the 
efficient frontier (EE) is generated by excluding all programs that lie above a line segment joining any 
two programs or joining any program to the origin (Bala & Zarkin, 2002). The EE will always begin at 
the origin and end at the most effective program (that program lying furthest to the right of the graph). 

The key to creating an EE is to begin at the origin and draw line segments to join programs lying as far 
right as possible until the program lying furthest to the right is reached; when a series of line segments are 
drawn in this manner, it is easy to visually observe any programs that lie above the EE. Programs 
comprising the EE correspond to those with minimized ACERs and ICERs; the greater an ACER or 
ICER, the steeper the slope of the line segment that connects a program to the origin or two programs to 
each other, respectively. In that respect, EEs identify dominance among treatment categories; programs 
excluded from the EE have ACERs or ICERs greater than those of other programs being considered for 
implementation. In other words, any program located above the EE indicates that it is either strictly or 
extendedly dominated (Bala & Zarkin, 2002). 

Figure 1 depicts the six lY treatment combinations’ per-child cost and effectiveness as measured by 
the Behar Total Problem Behavior Score. Per-child costs are displayed along the Y-axis with per-child 
effectiveness (represented by standardized pre-post test difference scores) displayed along the X-axis. 

(The control condition is excluded from the figure because results indicated that behavior of control group 


294 


JEIBI 


3 . 4 - 4.1 


children worsened over the duration of the studies). An EF connects the CT, CT+TT, and CT+PT 
treatment categories to one another. Because they do not lie along the EF, the PT, PT+TT, and 
CT+PT+TT treatment categories represent either strictly or extendedly dominated treatment categories. In 
this scenario, all three categories are strictly dominated; two treatments exist (CT and CT+TT) that are 
both cheaper and more effective than the three treatment categories excluded from the EF. 

Figure 2 depicts each treatment categories’ per-child cost and effectiveness measured as by the 
DPICS-R Negative Child Behavior Score. (Again, data for the control condition are excluded from the 
figure because results indicated that the behavior of control group children worsened over the duration of 
the studies). In this scenario, the EF consists of only one line segment joining CT+PT+TT to the origin; 
because this line segment connects a program to the origin, it represents the ACER for CT+PT+TT. 

Hence, CT+PT+TT has the lowest ACER (the line segment with the least steep slope that can be drawn 
from the origin to a program) when effectiveness is measured using the DPICS-R Negative Behavior 
outcome variable. Figure 2 indicates that the CT, PT, CT+TT, PT+TT, and CT+PT treatment categories 
are either strictly or extendedly dominated because they lie above the EF; however, only PT+TT is strictly 
dominated by another treatment combination; compared to PT+TT, PT produces a higher per-child 
effectiveness for a lower per-child cost. That is, even though PT is, itself, an extendedly dominated 
category, it strictly dominates PT+TT by producing greater gains in effectiveness at lower costs. 

Figure I : Treatment Category Per-Child Cost and Effectiveness as Measured by the Behar Total Problem Behavior 

Score 



(Average Standardized Pre-Post Test Difference Score) 


Figure 1: Per-child cost and effectiveness data for the six experimental Incredihle Years treatment comhinations are 
plotted. Standardized pre-post test difference scores are measured hy the Behar Total Problem Behavior Score. The 
efficiency frontier excludes PT, PT-FTT, and CT+PT-I-TT, indicating that these are dominated treatment categories; of 
these, all three are strictly dominated (both by CT and CT+TT). 
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Fipre 2: Treatment Category Per-Child Cost and Effectiveness as Measured by the Combined 5-Item DPICS-R 
Negative Child Behavior Score 



Per-Child Effectiveness 

(Average Standardized Pre-Post Test Difference Score) 


Figure 2: Per-child cost and effectiveness data for the six experimental Incredible Years treatment combinations are 
plotted. Standardized pre-post test difference scores are measured by the Combined 5-Item DPICS-R Negative Child 
Behavior Score. Because they lie above the efficiency frontier, CT, CT-l-TT, PT, PT-I-TT, and CT-I-PT are dominated 
treatment categories. Of these PT-l-TT is strictly dominated (by PT). The remaining treatment categories excluded from 
the efficiency frontier - CT, CT-ETT, PT, and CT-EPT are extendedly dominated. 


Under explicit budget constraints, the goal is to select the treatment combination producing the 
greatest gains in effectiveness at a price less than or equal to D (the total dollar amount available per- 
child). When working under explicit budget constraints, agencies should first divide the available budget 
by the number of individuals to be treated within the target population; this will yield the per-person 
available budget (D). Next, health decision makers should plot the horizontal line Y=D so that it 
intersects with the EF. Below, the implementation decision process is outlined for three hypothetical 
explicit budget constraint scenarios. Agencies were hypothetically assumed to have the following per- 
child funds available for behavioral intervention: (1) $2,500 per child; (2) $1,250 per child; and (3) less 
than $1,164 per child (the per-child cost of CT, the cheapest treatment category considered here). 

Behar Total Problem Behavior Score. Under the hypothetical explicit budget constraint of D=$2,500, 
if all children are to receive the same treatment, health decision makers should implement CT-ETT; this 
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category represents the most effective treatment with per-child costs less than or equal to D=$2,500 (see 
Figure 1). Flowever, if children may receive different treatments, health decision makers are able to 
consult the EF to determine the combination of programs that would maximize benefits while remaining 
within per-child budget constraints. When the horizontal line Y=$2,500 is plotted, it intersects the EF at a 
point along the line segment connecting programs CT+TT and CT+PT. If children may receive different 
treatments, health decision makers should implement both CT+TT and CT+PT in the following line 
segment proportions: XB/AB receive treatment A and AX/AB receive treatment B, where letter A 
represents program CT+TT, letter B represents program CT+PT, and letter X represents the point at 
which the horizontal line Y=$2,500 intersects the EF (Bala & Zarkin, 2002). The point X, located on the 
EF between CT+TT and CT+PT, corresponds to a per-child cost equal to D, but with a higher per-child 
effectiveness than CT+TT; thus, point X represents maximized per-child effectiveness for a per-child cost 
of D. To maximize benefits without exceeding budget constraints, agencies can implement CT+TT and 
CT+PT in combination, so that on average, per-child cost and effectiveness correspond to those of point 
X. (The ability to implement two programs ensures that agencies come as close as possible to reaching 
per-child budget constraints without ever exceeding those constraints. However, when faced with the 
decision of whether to implement a combination of programs, agencies may wish to consider the logistics 
and morality of providing more than one type of treatment to a target population; if implementing more 
than one program proves to be difficult, agencies may be better off implementing the one program that 
produces the greatest benefits at a per-child cost cheaper than D.) 

If an agency is able to spend $1,250 per child, and all children must receive the same treatment, 
health decision makers should implement CT. If children may receive different treatments, agencies are 
able to maximize per-child effectiveness while adhering to budget constraints by implementing a 
combination of CT and CT+TT. Using the proportions described above, XB/AB proportion of target 
children would receive CT and AX/AB proportion of children would receive CT+TT, where A represents 
CT, B represents CT+TT and X represents the point at which Y=$ 1,250 intersects the EF. Again, 
agencies should consider their unique situations to determine if implementing a combination of two 
programs would be a preferable intervention strategy. 

Finally, if an agency is not able to spend at least $1,164 per child - the per-child cost of the cheapest 
treatment category (CT) - two options remain. First, an agency could move forward, implement CT, and 
treat as many children as possible until funds are exhausted; in this scenario, the agency would be forced 
to treat fewer children than originally planned. Or, second, an agency could attempt to locate another type 
of behavioral intervention with similar goals for a per-child cost lower than D. 

The Behar Total Problem Behavior Score data reflect another interesting outcome: while the per-child 
treatment costs of CT, CT+TT, and CT+PT differ substantially ($1,164, $1,454, and $2,713, 
respectively), their per-child effectiveness (as measured by the Behar Total Problem Behavior Score) does 
not (0.52, 0.53, and 0.55 standard deviations decrease in negative child behavior, respectively). Therefore, 
if an agency has a hypothetical explicit budget constraint of $2,800 per child, and teacher skills training 
rather than parent skills training is the desired focus of the intervention, it should strongly consider 
implementing CT+TT rather than CT+PT (the most effective treatment category with per-child treatment 
costs less than or equal to D=$2,800); if an agency decided to implement CT+TT rather than CT+PT, it 
would be able to treat substantially more children for a very slight trade-off in effectiveness, while also 
narrowing in more accurately on the treatment goals of the agency. If, however, parent skills training 
(rather than teacher skills training) is the major focus of the intervention, health decision makers should 
consider implementing CT+PT because it more appropriately addresses the specific needs of target 
children, while still meeting the financial requirement of a program with per-child costs lower than D. 

DPICS-R Negative Child Behavior Score. It is interesting to note that a combination of programs is 
not advised under this scenario (see Figure 2) because the EF consists of only one program (CT+PT+TT); 
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therefore, the line segment proportion rule deseribed above would involve providing target ehildren with 
a combination of CT+PT+TT or no treatment (corresponding to the origin) to reach the per-child cost and 
effectiveness corresponding to point X (where the horizontal line Y=D intersects the EF). As this is most 
likely not a preferable intervention strategy, agencies should seek to implement the one program that 
provides the greatest per-child gains while remaining within budget constraints. 

Under the hypothetical explicit budget constraint of Y=$2,500 per child, health decision makers 
should implement PT only. Although PT does not lie along the EF, this treatment category represents the 
most effective treatment category with a per-child cost less than D=$2,500. (Note: if we were to analyze 
these results assuming an agency is limited by implicit (rather than explicit) budget constraints, the fact 
that PT is not located along the efficient frontier would factor into the decisions regarding implementation 
approaches. (Please refer to Bala and Zarkin (2002) for an overview of the implementation decision 
process for agencies limited by implicit budget constraints.) If an agency is able to spend $1,250 per 
child, health decision makers should implement CT. This treatment category maximizes child gains at a 
per-child cost less than D=$ 1,250. Finally, if an agency is not able to spend $1,164 per child (i.e., D < 

$1,1 64), agencies can either implement CT and treat fewer children than originally planned or search for 
another behavioral intervention with similar goals for a per-child cost lower than D. 

The use of multiple outcome measures with CEA. The pooled lY Series data used for this study 
included seven outcome measures of treatment effectiveness (however, based on the objectives of this 
article, only two outcome measures were included for illustrative purposes). If health decision makers 
have per-child treatment effectiveness data for multiple child outcome measures, they should plot the line 
Y=D across graphs for each outcome variable. If a treatment category is selected for implementation 
across more than one outcome variable, health decision makers are increasingly confident in their 
decision to implement that particular treatment. For this paper, post-test difference scores for seven 
outcome variables were analyzed (results not presented here); five of these seven variables measured 
children’s externalizing behaviors consistent with CD. Across four of the five externalizing measures, the 
PT+TT treatment category was immediately identified as a strictly dominated treatment and, therefore, 
eliminated from the pool of potential treatment approaches. On the other hand, the CT+PT+TT treatment 
category was not identified as strictly dominated across three of the five externalizing measures 
examined, indicating its strong potential as a cost-effective implementation strategy for the behavioral 
intervention. Such information, combined with information regarding agencies’ treatment goals and target 
populations, will aid health decision makers when selecting program implementation formats. 

Discussion 

Additional points of consideration. It is important to note that decisions to implement a particular 
combination of treatment components should consider target children’s unique behavioral symptoms. For 
example, if a child’s behavior problems are confined to the home setting, it seems illogical for health 
decision makers to implement TT only (unless outcome data showed that TT was successful at reducing 
negative behaviors in the home setting). Therefore, while per-child cost and effectiveness data, as well as 
budget constraint information, guide the CEA decision-making process, it is important to consider 
children’s specific needs as well. If a child displays pervasive behavior problems across multiple contexts, 
a treatment combination that targets these multiple foci within a child’s life may be preferred despite its 
higher per-child costs. Similarly, if a child’s negative behavior is confined to one setting (e.g., home or 
school), spending funds on a treatment combination that addresses multiple contexts may be considered 
excessive. As pointed out by Hester and Kaiser (1998), treatment effectiveness should be assessed using 
conceptually-driven outcome measures. For this reason, it is important for health decision makers to 
conduct CEA using appropriate outcome data. If health decision makers are primarily interested in 
reducing children’s negative behaviors at home, they should consider performing CEA using outcome 
data gathered from a home observational measure. On the other hand, if the prime goal is to reduce 
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negative behaviors in the elassroom, they should eonsider evaluating treatment effectiveness as measured 
using teacher-report or classroom observational measures. 

In addition to children’s needs, treatment decisions should take into account familial aspects. A 
family’s comfort and satisfaction with, and probability of adherence to, a treatment combination, as well 
as any quality of life adjustments that may be brought about by participation in such a program, should be 
considered prior to implementation. Furthermore, it is important to note that while a program may be 
considered cost-effective, whether or not agencies and schools choose to adopt these interventions will 
depend on their willingness and ability to pay for CD interventions, treatments, and training. 

Finally, when choosing an implementation strategy, health agencies should also consider secondary 
populations served by each treatment combination. For example, combinations that include PT may 
provide additional benefits for those siblings of target children. Likewise, combinations that incorporate 
TT impact not only the target child, but the entire classroom of children and any future children with 
behavior problems that this teacher interacts with. 

Policy implications. This paper used data from the lY Series to demonstrate cost-effectiveness 
methodology. The lY Series is an evidence-based, multi-component intervention designed to treat and 
prevent early-onset CD among very young children; it has been adopted by hundreds of agencies seeking 
to provide children and their families with a comprehensive behavioral intervention. As agencies continue 
to implement the lY Series both within the United States and abroad, they must determine which 
implementation strategy maximizes children’s gains while remaining within agencies’ budget constraints. 
CEA helps agencies implementing the lY Series to spend their resources in the most efficient manner, 
thus producing the largest possible effect size in the largest possible child population. The concepts and 
methodology presented in this article will be relevant for any public or private agency attempting to select 
a cost-effective intervention from a series of potential delivery approaches. 

The CEA data generated in this paper have policy implications within the realm of juvenile justice 
systems, public school systems, and child welfare systems. Given the large literature detailing the 
enormous societal costs resulting from negative behaviors associated with CD, investigations into the 
economic appropriateness of CD prevention may yield new efforts to re-organize financial resources for 
widespread implementation of behavioral interventions. Again, it is important to realize that cost- 
effectiveness data should be combined with information regarding the specific needs of target children. 
Health decision makers should strive to match the nature of children’s pervasiveness of negative behavior 
with a treatment program that targets the desired contexts while minimizing costs. By doing so, health 
decision makers are able to select the most cost-effective treatment combination likely to be effective for 
a particular group of young children. Additionally, health decision makers should consider that even the 
most expensive lY implementation strategy - CT+PT+TT - is considerably low in cost compared with 
the cost of incarceration, substance use, crime, and other negative outcomes that may result when children 
displaying early-onset conduct problems are not treated effectively (Jones et al., 2002). 

Conclusions and Limitations 

CEA is a useful analytic approach to gauge how competing programs do or do not maximize gains 
while minimizing costs. CEA is also practical for evaluating treatments and interventions targeted toward 
very young children. In such interventions, data involving crime, substance use, school drop-out rates, 
and other negative public health outcomes are not available for a number of years post-test. Whereas 
typical benefit cost-analysis would require prospective data, CEA provides immediate financial estimates 
by incorporating effectiveness measures without assigning monetary values. 


299 


JEIBI 


3 . 4 - 4.1 


The CEA methodology presented in this paper also allows health decision makers to examine a 
treatment’s effectiveness according to a specific type of outcome. If health decision makers are most 
interested in a particular result, they can perform CEA to gain insight into treatment effectiveness for the 
primary outcome of interest. For example, if an agency considers parent skills training to be the most 
important proximal outcome, and child skills training is distal to parent change, decision makers are able 
to perform CEA analyses using parent behavior outcome data (i.e. parenting style or parent discipline 
outcome data). Therefore, CEA analysis aids decision makers not only in choosing a treatment 
combination that maximizes gains while minimizing costs, but in choosing a combination that maximizes 
specifically desired gains while minimizing costs. Despite its versatility and utility, however, CEA is best 
combined with additional information specific to an agency’s unique situation for determining the “best” 
approach for treatment implementation. 

The data employed in this study present a small number of methodological limitations. The first 
involves the design of the clinic-based studies that generated the child outcome data considered here. 
While participants were randomly assigned to treatment combinations, they were spread across multiple 
cohorts and comprise relatively small groups. It is not clear, however, how one could obtain the necessary 
information in any other way: randomly assigning large numbers of participants to multiple treatment 
combinations would be enormously expensive. As a result, some of the comparisons across treatment 
groups may confound treatment effects with cohort effects. It is important, however, to note that 
participants were drawn from a single geographic area and were identified using the same eligibility 
criteria. 

As a result of the small sample sizes, considerable uncertainty surrounds the choice of a cost-effective 
program. Regardless of small sample sizes, however, a policy maker still may have to decide which 
treatment to implement; therefore, even data based on small samples are arguably more helpful to the 
decision-making process than a complete absence of such data. An informed decision, however, must 
also reflect the uncertainty surrounding that information. 

Another limitation involves the generalizability of the results. The cost-effectiveness estimates 
generated here are based on immediate post-test behavioral outcomes of children age 3-8; therefore, 
estimates may not reflect the experiences of older children diagnosed with CD who may take part in the 
lY Series. Similarly, as noted, nearly all of the study’s participants were Caucasian. Whether and how the 
cost-effective choice of treatments would differ for minority youth is an area for future research. 

Future research should examine if and how strict dominance among treatment combinations varies 
when using immediate follow-up outcome data versus one-year follow-up outcome data; incorporating 
one-year follow-up assessments would offer insight into treatments’ cost-effectiveness over time. Indeed, 
Craig and Digout (2003) highlight the need for longer follow-up following the completion of 
interventions. Furthermore, Miller et al. (2003) argue that cost-effectiveness ratio point estimates based 
on average cost and outcome differences among treatments (i.e. ACERs) fail to capture uncertainty within 
the data. To summarize the entire data distribution, cost-effectiveness acceptability curves - an advanced 
technique within the realm of CEA - should be generated (Miller et al., 2003). This methodology 
represents an avenue of future research involving lY data. 

Finally, by using CEA alone to make judgments regarding the implementation of mental health 
interventions, health decision makers may fail to consider societal viewpoints regarding treatment worth. 
As noted in Jensen, Garcia, Glied, Crowe, Foster, Schlander, et al. (2005), there are no monetary 
threshold values that serve as guidelines for determining the precise cost at which a combination may be 
considered “cost-effective”; furthermore, announcing that one treatment combination is “cost-effective” 
while another is “not cost-effective” may imply judgment about societal values concerning the worth of 
treatment benefits. Therefore, while CEA offers insight into the financial feasibility of competing 
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programs, a broader societal perspective on the treatment and prevention of CD should also be considered 

when selecting an intervention implementation approach. 
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